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Amendments to the Claims: 

This listing of claims will replace all prior versions, and listings, of claims in the application: 
Listing of Claims: 

1 . (Original) A method of displaying multimedia information stored in a 
multimedia document on a display, the multimedia information comprising information of a 
plurality of types including information of a first type and information of a second type, the 
method comprising: 

displaying a graphical user interface (GUI) on the display; 

displaying, in a first area of the GUI, a representation of the multimedia 
information stored by the multimedia document, the displayed representation of the multimedia 
information comprising a representation of information of the first type and a representation of 
information of the second type; 

displaying a first lens covering a first portion of the first area; and 

displaying, in a second area of the GUI, a representation of multimedia 
information displayed in the first portion of the first area, the representation of multimedia 
information displayed in the second area comprising a portion of the representation of 
information of the first type covered by the first lens and a portion of the representation of 
information of the second type covered by the first lens. 

2. (Original) The method of claim 1 wherein displaying the representation of 
the multimedia information stored by the multimedia document in the first area of the GUI 
comprises: 

displaying a first thumbnail image in the first area of the GUI, the first thumbnail 
image comprising the representation of information of the first type; and 

displaying a second thumbnail image in the first area of the GUI, the second 
thumbnail image comprising the representation of information of the second type. 
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3. (Original) The method of claim 1 wherein displaying, in the second area 
of the GUI, the representation of multimedia information displayed in the first portion of the first 
area comprises: 

displaying the portion of the representation of information of the first type 
covered by the first lens in a first panel in the second area of the GUI; and 

displaying the portion of the representation of information of the second type 
covered by the first lens in a second panel in the second area of the GUI. 

4. (Original) The method of claim 1 wherein displaying, in the second area 
of the GUI, the representation of multimedia information displayed in the first portion of the first 
area comprises: 

determining a first time and a second time associated with the first lens; 

displaying, in the second area of the GUI, a representation of information of the 
first type occurring between the first time and the second time associated with the first lens; and 

displaying, in the second area of the GUI, a representation of information of the 
second type occurring between the first time and the second time associated with the first lens. 

5. (Original) The method of claim 1 further comprising: 

receiving user input moving the first lens to cover a second portion of the first 

area; and 

responsive to the user input, automatically changing the information displayed in 
the second area of the GUI such that the representation of multimedia information displayed in 
the second area of the GUI corresponds to the representation of multimedia information included 
in the second portion of the first area. 

6. (Original) The method of claim 1 further comprising: 
displaying a second lens covering a first portion of the second area; and 
displaying, in a third area of the GUI, a representation of multimedia information 

corresponding to the first portion of the second area, the representation of multimedia 
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information displayed in the third area comprising a portion of the representation of information 
of the first type covered by the second lens and a portion of the representation of information of 
the second type covered by the second lens. 

7. (Original) The method of claim 6 wherein displaying, in the third area of 
the GUI, the representation of multimedia information corresponding to the first portion of the 
second area comprises: 

determining a first time and a second time associated with the second lens; 

displaying, in the third area of the GUI, a representation of information of the first 
type occurring between the first time and the second time associated with the second lens; and 

displaying, in the third area of the GUI, a representation of information of the 
second type occurring between the first time and the second time associated with the second lens. 

8. (Original) The method of claim 6 wherein: 

displaying the representation of the multimedia information stored by the 
multimedia document in the first area of the GUI comprises: 

displaying a first thumbnail image in the first area of the GUI, the first 
thumbnail image comprising the representation of information of the first type; and 

displaying a second thumbnail image in the first area of the GUI, the 
second thumbnail image comprising the representation of information of the second type; 

displaying the representation of multimedia information displayed in the first 
portion of the first area in the second area of the GUI comprises: 

displaying the portion of the representation of information of the first type 
covered by the first lens in a first panel in the second area of the GUI; and 

displaying the portion of the representation of information of the second 
type covered by the first lens in a second panel in the second area of the GUI; and 

displaying the representation of multimedia information corresponding to the first 
portion of the second area in the third area of the GUI comprises: 
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displaying the representation of information of the first type corresponding 
to the first portion of the second area of the GUI in a first sub-area of the third area of the GUI; 
and 

displaying the representation of information of the second type 
corresponding to the first portion of the second area of the GUI in a second sub-area of the third 
area of the GUI. 

9. (Original) The method of claim 6 further comprising: 

receiving a user input moving the second lens to cover a second portion of the 
second area; and 

responsive to the user input, automatically changing the information displayed in 
the third area of the GUI such that the representation of multimedia information displayed in the 
third area of the GUI corresponds to the representation of the multimedia information included in 
the second portion of the second area. 

10. (Original) The method of claim 6 further comprising: 

receiving a user input moving the first lens to cover a second portion of the first 

area; and 

responsive to the user input, automatically: 

changing the information displayed in the second area of the GUI such 
that the representation of multimedia information displayed in the second area of the GUI 
corresponds to the representation of multimedia information included in the second portion of 
the first area; and 

changing the information displayed in the third area of the GUI such that 
the representation of multimedia information displayed in the third area of the GUI corresponds 
to the representation of the multimedia information included in the second portion of the second 
area. 

1 1 . (Original) The method of claim 6 further comprising: 
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displaying a sub-lens covering a portion of the first area of the GUI corresponding 
to the first portion of the second area of the GUI covered by the second lens. 

12. (Original) The method of claim 1 1 further comprising: 

receiving a user input moving the second lens to cover a second portion of the 
second area; and 

responsive to the user input, automatically changing a position of the sub-lens to 
cover a portion of the first area of the GUI corresponding to the second portion of the second 
area. 

13. (Original) The method of claim 1 wherein: 

the information of the first type corresponds to video information; and 
the representation of the information of the first type comprises one or more video 
keyframes extracted from the video information. 

14. (Original) The method of claim 13 wherein: 

the information of the second type corresponds to audio information; and 
the representation of information of the second type comprises text information 
obtained from transcribing the audio information. 

15. (Original) The method of claim 13 wherein: 

the information of the second type corresponds to closed-caption (CC) text 
information; and 

the representation of information of the second type comprises text information 
included in the CC text information. 

16. (Original) The method of claim 1 further comprising: 
receiving information indicating a user-specified concept of interest; and 
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analyzing the multimedia information stored in the multimedia document to 
identify one or more locations in the multimed ia information that are relevant to the user- 
specified concept of interest; 

wherein displaying the representation of multimedia information in the first area 
of the GUI comprises annotating the one or more locations in the multimedia information that 
are relevant to the user-specified concept of interest; and 

wherein displaying, in the second area of the GUI, a representation of multimedia 
information displayed in the first portion of the first area comprises annotating the one or more 
locations in the multimedia information that are relevant to the user-specified concept of interest 
and that are located in the first portion of the first area. 

17. (Original) The method of claim 1 further comprising: 

receiving input indicating selection of a portion of the multimedia information 
occurring between a first time and a second time; and 

performing a first operation on the portion of the multimedia information 
occurring between a first time and a second time. 

18. (Original) A method of displaying multimedia information stored in a 
multimedia document on a display, the multimedia information comprising information of a first 
type and information of a second type, the method comprising: 

displaying a graphical user interface (GUI) on the display; 

displaying, in a first area of the GUI, a representation of the multimedia 
information stored by the multimedia document occurring between a start time (t s ) and an end 
time (t e ) associated with the multimedia document, the displayed representation of the 
multimedia information comprising a representation of information of the first type occurring 
between t s and t e and a representation of information of the second type occurring between t s 
and t e , where (t e > t s ); 
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displaying a first lens emphasizing a portion of the first area of the GUI, the 
portion of the first area emphasized by the first lens comprising a representation of multimedia 
information occurring between a first time (ti ) and a second time (12), where (t s t\ < t e ); an d 

displaying, in a second area of the GUI, the representation of multimedia 
information occurring between t^ and t2, the representation of multimedia information displayed 
in the second area comprising a representation of information of the first type occurring between 
t\ and t2 and a representation of information of the second type occurring between t\ and t2- 

19. (Original) The method of claim 18 further comprising: 

displaying a second lens emphasizing a portion of the second area of the GUI, the 
portion of the second area emphasized by the second lens comprising a representation of 
multimedia information occurring between a third time O3) and a fourth time O4), where (tj 13 < 
14 12); and 

displaying, in a third area of the GUI, the representation of multimedia 
information occurring between t3 and 14, the representation of multimedia information displayed 
in the third area comprising a representation of information of the first type occurring between t3 
and 14 and a representation of information of the second type occurring between t3 and t4. 

20. (Original) The method of claim 19 further comprising: 

changing the position of the first lens in response to user input such that the first 
lens emphasizes a portion of the first area of the GUI comprising a representation of multimedia 
information occurring between a fifth time (t 5) and a sixth time (tg), where (t s t5 < tg t e ), (15 
t\ ), and (tg t2); and 

responsive to the change in the position of the first lens, automatically displaying, 
in the second area of the GUI, the representation of multimedia information occurring between t5 
and tg, the representation of multimedia information displayed in the second area comprising a 
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representation of information of the first type occurring between t5 and tg and a representation of 
information of the second type occurring between t5 and tg. 

21. (Original) The method of claim 19 further comprising: 

changing the position of the second lens in response to user input such that the 
second lens emphasizes a portion of the second area of the GUI comprising a representation of 
multimedia information occurring between a fifth time (t5) and a sixth time (tg), where (t^ t5 < 
t6 t2), (t5 t 3 ), and (t 6 t 4 ); and 

responsive to the change in the position of the second lens, automatically 
displaying, in the third area of the GUI, the representation of multimedia information occurring 
between t5 and tg, the representation of multimedia information displayed in the third area 
comprising a representation of information of the first type occurring between t5 and tg and a 
representation of information of the second type occurring between t5 and t£. 

22. (Original) The method of claim 19 further comprising: 
displaying a third lens emphasizing a portion of the first area of the GUI 

comprising a representation of multimedia information occurring between t 3 and t 4 

23. (Original) The method of claim 22 further comprising: 

changing the position of the second lens in response to user input such that the 
second lens emphasizes a portion of the second area of the GUI comprising a representation of 
multimedia information occurring between a fifth time (15) and a sixth time (tg), where (t\ ts, < 
t6 t2), (t5 t 3 ), and (t 6 t 4 ); and 

responsive to the change in the position of the second lens, automatically 
changing the position of the third lens such that the third lens emphasizes a portion of the first 
area of the GUI comprising a representation of multimedia information occurring between t5 and 

t6. 
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24. (Original) The method of claim 18 wherein: 
the information of the first type is video information; 
the information of the second type is audio information; 

the representation of the information of the first type comprises one or more video 
keyframes extracted from the video information; and 

the representation of information of the second type comprises text information 
obtained from transcribing the audio information. 

25. (Original) The method of claim 18 wherein: 
the information of the first type is video information; 

the information of the second type is closed-caption (CC) text information; 

the representation of the information of the first type comprises one or more video 
keyframes extracted from the video information; and 

the representation of the information of the second type comprises text 
information included in the CC text information. 

26. (Original) The method of claim 18 further comprising: 
receiving information indicating a first topic; and 

analyzing the multimedia information stored in the multimedia document to 
identify one or more locations in the multimedia information that are relevant to the first topic; 

wherein displaying the representation of the multimedia information stored by the 
multimedia document occurring between t s and t e in the first area of the GUI comprises 
highlighting the one or more locations in the multimedia information displayed in the first area 
of the GUI; and 

wherein displaying the representation of multimedia information occurring 
between t\ and t2 in the second area of the GUI comprises highlighting the one or more locations 
in the multimedia information that occur between times ti and t2- 

27. (Original) The method of claim 18 further comprising: 
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receiving input indicating selection of a portion of the multimedia information 
occurring between a selection start time and a selection end time; and 

performing a first operation on the portion of the multimedia information 
occurring between the selection start time and the selection end time. 

28. (Original) A method of displaying multimedia information stored in a 
multimedia document on a display, the multimedia information comprising video information 
and information of a first type, the method comprising: 

displaying a graphical user interface (GUI) on the display; 

displaying, in a first section of a first area of the GUI, a first set of one or more 
video keyframes extracted from the video information occurring between a start time (t s ) and an 
end time (t e ) associated with the multimedia document, where (t e > t s ); 

displaying, in a second section of the first area of the GUI, text information 
corresponding to the information of the first type occurring between t s and t e ; 

displaying a first lens emphasizing a portion of the first section of the first area 
occurring between a first time (ti ) and a second time 02) and a portion of the second section of 
the first area occurring between t^ and t2, the emphasized portion of the first section of the first 
area comprising a second set of one or more video keyframes extracted from the video 
information occurring between t^ and t2, the emphasized portion of the second section of the 
first area comprising text information corresponding to information of the first type occurring 
between t \ and t2, wherein the second set of one or more keyframes is a subset of the first set of 
one or more keyframes and (t s ti < t2 t e ); 

displaying the second set of one or more keyframes in a first section of a second 
area of the GUI; and 

displaying text information corresponding to the information of the first type 
occurring between t \ and t2 in a second section of the second area of the GUI. 

29. (Original) The method of claim 28 further comprising: 
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displaying a second lens emphasizing a portion of the first section of the second 
area and a portion of the second section of the second area, the emphasized portion of the first 
section of the second area comprising a third set of one or more video keyframes extracted from 
the video information occurring between a third time (t3) and a fourth time O4), the emphasized 
portion of the second section of the second area comprising text information corresponding to 
information of the first type occurring between t3 and wherein the third set of one or more 
video keyframes is a subset of the second set of one or more video keyframes and (t\ ti, < t4 

t 2 ); 

displaying a keyframe from the third set of one or more keyframes in a first 
section of a third area of the GUI; and 

displaying text information corresponding to the information of the first type 
occurring between t3 and 14 in a second section of the third area of the GUI. 

30. (Original) The method of claim 28 further comprising: 
displaying a second lens emphasizing a portion of the first section of the second 
area and a portion of the second section of the second area, the emphasized portion of the first 
section of the second area comprising a third set of one or more video keyframes extracted from 
the video information occurring between a third time (ti,) and a fourth time (14), the emphasized 
portion of the second section of the second area comprising text information corresponding to 
information of the first type occurring between t3 and t4, wherein the third set of one or more 
video keyframes is a subset of the second set of one or more video keyframes and (t^ t3 < t4 

t 2 ); 

outputting video information starting from 13 or from t4 or from a time between t3 
and t4 in a first section of a third area of the GUI; and 

displaying text information corresponding to the information of the first type 
occurring between t3 and 14 in a second section of the third area of the GUI. 
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3 1 . (Original) The method of claim 28 wherein the information of the first 
type is audio information, and the text information corresponding to the information of the first 
type is obtained from transcribing the audio information. 

32. (Original) The method of claim 28 wherein the information of the first 
type is closed-caption (CC) text information, and the text information corresponding to the 
information of the first type is extracted from the CC text information. 

33. (Original) The method of claim 28 wherein the multimedia information 
stored by the multimedia document further comprises slides information, the method comprising: 

displaying, in a third section of the first area of the GUI, a first set of one or more 
slides extracted from the slides information occurring between t s and t e , wherein the first lens 
emphasizes a portion of the third section of the first area comprising a second set of one or more 
slides extracted from the slides information occurring between t\ and t2, the second set of one or 
more slides is a subset of the first set of one or more slides; and 

displaying the second set of one or more slides in a third section of the second 
area of the GUI. 

34. (Original) The method of claim 33 further comprising: 

displaying a second lens emphasizing a portion of the first section of the second 
area, a portion of the second section of the second area, and a portion of the third section of the 
second area, the emphasized portion of the first section of the second area comprising a third set 
of one or more video keyframes extracted from the video information occurring between a third 
time (t3) and a fourth time (14), the emphasized portion of the second section of the second area 
comprising text information corresponding to information of the first type occurring between t3 
and 14, the emphasized portion of the third section of the second area comprising a third set of 
one or more slides extracted from the slides information occurring between 13 and t4, wherein 
the third set of one or more video keyframes is a subset of the second set of one or more video 
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keyframes, the third set of one or more slides is a subset of the second set of one or more slides, 
and fa t 3 <t 4 t 2 ); 

displaying at least one keyframe from the third set of one or more video 
keyframes in a first section of a third area of the GUI; 

displaying text information corresponding to the information of the first type 
occurring between t 3 and 14 in a second section of the third area of the GUI; and 

displaying at least one slide from the third set of one or more slides in a third 
section of the third area of the GUI. 

35. (Original) The method of claim 28 wherein the multimedia information 
stored by the multimedia document further comprises whiteboard images information, the 
method comprising: 

displaying, in a third section of the first area of the GUI, a first set of one or more 
whiteboard images extracted from the whiteboard images information occurring between t s and 
t e , wherein the first lens emphasizes a portion of the third section of the first area comprising a 
second set of one or more whiteboard images extracted from the whiteboard images information 
occurring between t\ and t 2 , the second set of one or more whiteboard images is a subset of the 
first set of one or more whiteboard images; and 

displaying the second set of one or more whiteboard images in a third section of 
the second area of the GUI. 

36. (Original) The method of claim 35 further comprising: 

displaying a second lens emphasizing a portion of the first section of the second 
area, a portion of the second section of the second area, and a portion of the third section of the 
second area, the emphasized portion of the first section of the second area comprising a third set 
of one or more video keyframes extracted from the video information occurring between a third 
time (t 3 ) and a fourth time (14), the emphasized portion of the second section of the second area 
comprising text information corresponding to information of the first type occurring between t 3 
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and t4, the emphasized portion of the third section of the second area comprising a third set of 
one or more whiteboard images extracted from the whiteboard images information occurring 
between t3 and t^, wherein the third set of one or more video keyframes is a subset of the second 
set of one or more video keyframes, the third set of one or more whiteboard images is a subset of 
the second set of one or more whiteboard images, and (t \ t^< t2); 

displaying at least one keyframe from the third set of one or more video 
keyframes in a first section of a third area of the GUI; 

displaying text information corresponding to the information of the first type 
occurring between t3 and t4 in a second section of the third area of the GUI; and 

displaying a whiteboard images from the third set of one or more whiteboard 
images in a third section of the third area of the GUI. 

37. (Original) A system for displaying multimedia information stored in a 
multimedia document on a display, the multimedia information comprising information of a 
plurality of types including information of a first type and information of a second type, the 
system comprising: 

means for displaying a graphical user interface (GUI) on the display; 

means for displaying, in a first area of the GUI, a representation of the multimedia 
information stored by the multimedia document, the displayed representation of the multimedia 
information comprising a representation of information of the first type and a representation of 
information of the second type; 

means for displaying a first lens covering a first portion of the first area; and 

means for displaying, in a second area of the GUI, a representation of multimedia 
information displayed in the first portion of the first area, the representation of multimedia 
information displayed in the second area comprising a portion of the representation of 
information of the first type covered by the first lens and a portion of the representation of 
information of the second type covered by the first lens. 
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38. (Original) A system for displaying multimedia information stored in a 
multimedia document on a display, the multimedia information comprising information of a first 
type and information of a second type, the system comprising: 

means for displaying a graphical user interface (GUI) on the display; 

means for displaying, in a first area of the GUI, a representation of the multimedia 
information stored by the multimedia document occurring between a start time (t s ) and an end 
time (t e ) associated with the multimedia document, the displayed representation of the 
multimedia information comprising a representation of information of the first type occurring 
between t s and t e and a representation of information of the second type occurring between t s 
and t e , where (t e > t s ); 

means for displaying a first lens emphasizing a portion of the first area of the 
GUI, the portion of the first area emphasized by the first lens comprising a representation of 
multimedia information occurring between a first time (t\) and a second time 02), where (t s t\ < 
t2 te); and 

means for displaying, in a second area of the GUI, the representation of 
multimedia information occurring between t\ and t2, the representation of multimedia 
information displayed in the second area comprising a representation of information of the first 
type occurring between t\ and t2 and a representation of information of the second type 
occurring between t\ and t2- 

39. (Original) A system for of displaying multimedia information stored in a 
multimedia document on a display, the multimedia information comprising video information 
and information of a first type, the system comprising: 

means for displaying a graphical user interface (GUI) on the display; 

means for displaying, in a first section of a first area of the GUI, a first set of one 
or more video keyframes extracted from the video information occurring between a start time (t s ) 
and an end time (t e ) associated with the multimedia document, where (t e > t s ); 
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means for displaying, in a second section of the first area of the GUI, text 
information corresponding to the information of the first type occurring between t s and t e ; 

means for displaying a first lens emphasizing a portion of the first section of the 
first area occurring between a first time (ti ) and a second time (t2) and a portion of the second 
section of the first area occurring between t\ and t2, the emphasized portion of the first section of 
the first area comprising a second set of one or more video keyframes extracted from the video 
information occurring between t^ and t2, the emphasized portion of the second section of the 
first area comprising text information corresponding to information of the first type occurring 
between t \ and t2, wherein the second set of one or more keyframes is a subset of the first set of 
one or more keyframes and (t s ti < t2 t e ); 

means for displaying the second set of one or more keyframes in a first section of 
a second area of the GUI; and 

means for displaying text information corresponding to the information of the first 
type occurring between t^ and t2 in a second section of the second area of the GUI. 

40. (Original) A computer program product stored on a computer-readable 
storage medium for displaying multimedia information stored in a multimedia document on a 
display, the multimedia information comprising information of a plurality of types including 
information of a first type and information of a second type, the computer program product 
comprising: 

code for displaying a graphical user interface (GUI) on the display; 

code for displaying, in a first area of the GUI, a representation of the multimedia 
information stored by the multimedia document, the displayed representation of the multimedia 
information comprising a representation of information of the first type and a representation of 
information of the second type; 

code for displaying a first lens covering a first portion of the first area; and 

code for displaying, in a second area of the GUI, a representation of multimedia 
information displayed in the first portion of the first area, the representation of multimedia 
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information displayed in the second area comprising a portion of the representation of 
information of the first type covered by the first lens and a portion of the representation of 
information of the second type covered by the first lens. 

41 . (Original) The computer program product of claim 40 wherein the code 
for displaying the representation of the multimedia information stored by the multimedia 
document in the first area of the GUI comprises: 

code for displaying a first thumbnail image in the first area of the GUI, the first 
thumbnail image comprising the representation of information of the first type; and 

code for displaying a second thumbnail image in the first area of the GUI, the 
second thumbnail image comprising the representation of information of the second type. 

42. (Original) The computer program product of claim 40 wherein the code 
for displaying, in the second area of the GUI, the representation of multimedia information 
displayed in the first portion of the first area comprises: 

code for displaying the portion of the representation of information of the first 
type covered by the first lens in a first panel in the second area of the GUI; and 

code for displaying the portion of the representation of information of the second 
type covered by the first lens in a second panel in the second area of the GUI. 

43. (Original) The computer program product of claim 40 wherein the code 
for displaying, in the second area of the GUI, the representation of multimedia information 
displayed in the first portion of the first area comprises: 

code for determining a first time and a second time associated with the first lens; 

code for displaying, in the second area of the GUI, a representation of information 
of the first type occurring between the first time and the second time associated with the first 
lens; and 
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code for displaying, in the second area of the GUI, a representation of information 
of the second type occurring between the first time and the second time associated with the first 
lens. 

44. (Original) The computer program product of claim 40 further comprising: 
code for receiving user input moving the first lens to cover a second portion of the 

first area; and 

code for responsive to the user input, automatically changing the information 
displayed in the second area of the GUI such that the representation of multimedia information 
displayed in the second area of the GUI corresponds to the representation of multimedia 
information included in the second portion of the first area. 

45. (Original) The computer program product of claim 40 further comprising: 
code for displaying a second lens covering a first portion of the second area; and 
code for displaying, in a third area of the GUI, a representation of multimedia 

information corresponding to the first portion of the second area, the representation of 
multimedia information displayed in the third area comprising a portion of the representation of 
information of the first type covered by the second lens and a portion of the representation of 
information of the second type covered by the second lens. 

46. (Original) The computer program product of claim 45 wherein the code 
for displaying, in the third area of the GUI, the representation of multimedia information 
corresponding to the first portion of the second area comprises: 

code for determining a first time and a second time associated with the second 

lens; 

code for displaying, in the third area of the GUI, a representation of information 
of the first type occurring between the first time and the second time associated with the second 
lens; and 
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code for displaying, in the third area of the GUI, a representation of information 
of the second type occurring between the first time and the second time associated with the 
second lens. 

47. (Original) The computer program product of claim 45 wherein: 

the code for displaying the representation of the multimedia information stored by 
the multimedia document in the first area of the GUI comprises: 

code for displaying a first thumbnail image in the first area of the GUI, the 
first thumbnail image comprising the representation of information of the first type; and 

code for displaying a second thumbnail image in the first area of the GUI, 
the second thumbnail image comprising the representation of information of the second type; 

the code for displaying the representation of multimedia information displayed in 
the first portion of the first area in the second area of the GUI comprises: 

code for displaying the portion of the representation of information of the 
first type covered by the first lens in a first panel in the second area of the GUI; and 

code for displaying the portion of the representation of information of the 
second type covered by the first lens in a second panel in the second area of the GUI; and 

the code for displaying the representation of multimedia information 
corresponding to the first portion of the second area in the third area of the GUI comprises: 

code for displaying the representation of information of the first type 
corresponding to the first portion of the second area of the GUI in a first sub-area of the third 
area of the GUI; and 

code for displaying the representation of information of the second type 
corresponding to the first portion of the second area of the GUI in a second sub-area of the third 
area of the GUI. 

48. (Original) The computer program product of claim 45 further comprising: 
code for receiving a user input moving the second lens to cover a second portion 

of the second area; and 
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responsive to the user input, code for automatically changing the information 
displayed in the third area of the GUI such that the representation of multimedia information 
displayed in the third area of the GUI corresponds to the representation of the multimedia 
information included in the second portion of the second area. 

49. (Original) The computer program product of claim 45 further comprising: 
code for receiving a user input moving the first lens to cover a second portion of 

the first area; and 

responsive to the user input, code for automatically: 

changing the information displayed in the second area of the GUI such 
that the representation of multimedia information displayed in the second area of the GUI 
corresponds to the representation of multimedia information included in the second portion of 
the first area; and 

changing the information displayed in the third area of the GUI such that 
the representation of multimedia information displayed in the third area of the GUI corresponds 
to the representation of the multimedia information included in the second portion of the second 
area. 

50. (Original) The computer program product of claim 45 further comprising: 
code for displaying a sub-lens covering a portion of the first area of the GUI 

corresponding to the first portion of the second area of the GUI covered by the second lens. 

5 1 . (Original) The computer program product of claim 50 further comprising: 
code for receiving a user input moving the second lens to cover a second portion 

of the second area; and 

responsive to the user input, code for automatically changing a position of the 
sub-lens to cover a portion of the first area of the GUI corresponding to the second portion of the 
second area. 

52. (Original) The computer program product of claim 40 wherein: 
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the information of the first type corresponds to video information; and 
the representation of the information of the first type comprises one or more video 
keyframes extracted from the video information. 

53. (Original) The computer program product of claim 52 wherein: 
the information of the second type corresponds to audio information; and 

the representation of information of the second type comprises text information 
obtained from transcribing the audio information. 

54. (Original) The computer program product of claim 52 wherein: 
the information of the second type corresponds to closed-caption (CC) text 

information; and 

the representation of information of the second type comprises text information 
included in the CC text information. 

55. (Original) The computer program product of claim 40 further comprising: 
code for receiving information indicating a user-specified concept of interest; and 
code for analyzing the multimedia information stored in the multimedia document 

to identify one or more locations in the multimedia information that are relevant to the user- 
specified concept of interest; 

wherein the code for displaying the representation of multimedia information in 
the first area of the GUI comprises code for annotating the one or more locations in the 
multimedia information that are relevant to the user-specified concept of interest; and 

wherein the code for displaying, in the second area of the GUI, a representation of 
multimedia information displayed in the first portion of the first area comprises code for 
annotating the one or more locations in the multimedia information that are relevant to the user- 
specified concept of interest and that are located in the first portion of the first area. 

56. (Original) The computer program product of claim 40 further comprising: 
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code for receiving input indicating selection of a portion of the multimedia 
information occurring between a first time and a second time; and 

code for performing a first operation on the portion of the multimedia information 
occurring between a first time and a second time. 

57. (Original) A computer program product stored on a computer-readable 
storage medium for displaying multimedia information stored in a multimedia document on a 
display, the multimedia information comprising information of a first type and information of a 
second type, the computer program product comprising: 

code for displaying a graphical user interface (GUI) on the display; 

code for displaying, in a first area of the GUI, a representation of the multimedia 
information stored by the multimedia document occurring between a start time (t s ) and an end 
time (t e ) associated with the multimedia document, the displayed representation of the 
multimedia information comprising a representation of information of the first type occurring 
between t s and t e and a representation of information of the second type occurring between t s 
and t e , where (t e > t s ); 

code for displaying a first lens emphasizing a portion of the first area of the GUI, 
the portion of the first area emphasized by the first lens comprising a representation of 
multimedia information occurring between a first time (ti ) and a second time O2), where (t s tj 
< t2 t e ); and 

code for displaying, in a second area of the GUI, the representation of multimedia 
information occurring between t^ and t2, the representation of multimedia information displayed 
in the second area comprising a representation of information of the first type occurring between 
t 1 and t2 and a representation of information of the second type occurring between t\ and t2- 

58. (Original) The computer program product of claim 57 further comprising: 
code for displaying a second lens emphasizing a portion of the second area of the 

GUI, the portion of the second area emphasized by the second lens comprising a representation 
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of multimedia information occurring between a third time O3) and a fourth time (14), where (t \ 
*3 < ty *2); anc * 

code for displaying, in a third area of the GUI, the representation of multimedia 
information occurring between t3 and 14, the representation of multimedia information displayed 
in the third area comprising a representation of information of the first type occurring between t3 
and 14 and a representation of information of the second type occurring between t3 and 14. 

59. (Original) The computer program product of claim 58 further comprising: 
code for changing the position of the first lens in response to user input such that 

the first lens emphasizes a portion of the first area of the GUI comprising a representation of 
multimedia information occurring between a fifth time O5) and a sixth time (tg), where (t s t5 < 
t 6 t e ), (t 5 t!), and (t 6 t 2 ); and 

responsive to the change in the position of the first lens, code for automatically 
displaying, in the second area of the GUI, the representation of multimedia information 
occurring between t5 and tg, the representation of multimedia information displayed in the 
second area comprising a representation of information of the first type occurring between 15 and 
tg and a representation of information of the second type occurring between t5 and tg. 

60. (Original) The computer program product of claim 58 further comprising: 
code for changing the position of the second lens in response to user input such 

that the second lens emphasizes a portion of the second area of the GUI comprising a 
representation of multimedia information occurring between a fifth time (t5) and a sixth time 
(t 6 ), where (ti t 5 < t 6 t 2 ), (t 5 t 3 ), and (t 6 t 4 ); and 

code for responsive to the change in the position of the second lens, automatically 
displaying, in the third area of the GUI, the representation of multimedia information occurring 
between t5 and tg, the representation of multimedia information displayed in the third area 
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comprising a representation of information of the first type occurring between t5 and tg and a 
representation of information of the second type occurring between t5 and tg. 

61 . (Original) The computer program product of claim 58 further comprising: 
code for displaying a third lens emphasizing a portion of the first area of the GUI 

comprising a representation of multimedia information occurring between t3 and 14 

62. (Original) The computer program product of claim 61 further comprising: 
code for changing the position of the second lens in response to user input such 

that the second lens emphasizes a portion of the second area of the GUI comprising a 
representation of multimedia information occurring between a fifth time (t5) and a sixth time 
(t 6 ), where (ti t 5 <t 6 t 2 ), (t 5 t 3 ),and(t 6 t 4 );and 

code for responsive to the change in the position of the second lens, automatically 
changing the position of the third lens such that the third lens emphasizes a portion of the first 
area of the GUI comprising a representation of multimedia information occurring between t5 and 

t6. 

63. (Original) The computer program product of claim 57 wherein: 
the information of the first type is video information; 

the information of the second type is audio information; 

the representation of the information of the first type comprises one or more video 
keyframes extracted from the video information; and 

the representation of information of the second type comprises text information 
obtained from transcribing the audio information. 

64. (Original) The computer program product of claim 57 wherein: 
the information of the first type is video information; 

the information of the second type is closed-caption (CC) text information; 
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the representation of the information of the first type comprises one or more video 
keyframes extracted from the video information; and 

the representation of the information of the second type comprises text 
information included in the CC text information. 

65. (Original) The computer program product of claim 57 further comprising: 
code for receiving information indicating a first topic; and 

code for analyzing the multimedia information stored in the multimedia document 
to identify one or more locations in the multimedia information that are relevant to the first topic; 

wherein the code for displaying the representation of the multimedia information 
stored by the multimedia document occurring between t s and t e in the first area of the GUI 
comprises code for highlighting the one or more locations in the multimedia information 
displayed in the first area of the GUI; and 

wherein the code for displaying the representation of multimedia information 
occurring between tj and t2 in the second area of the GUI comprises code for highlighting the 
one or more locations in the multimedia information that occur between times t\ and t2- 

66. (Original) The computer program product of claim 57 further comprising: 
code for receiving input indicating selection of a portion of the multimedia 

information occurring between a selection start time and a selection end time; and 

code for performing a first operation on the portion of the multimedia information 
occurring between the selection start time and the selection end time. 

67. (Original) A computer program product stored on a computer-readable 
storage medium for displaying multimedia information stored in a multimedia document on a 
display, the multimedia information comprising video information and information of a first 
type, the computer program product comprising: 

code for displaying a graphical user interface (GUI) on the display; 
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code for displaying, in a first section of a first area of the GUI, a first set of one or 
more video keyframes extracted from the video information occurring between a start time (t s ) 
and an end time (t e ) associated with the multimedia document, where (t e > t s ); 

code for displaying, in a second section of the first area of the GUI, text 
information corresponding to the information of the first type occurring between t s and t e ; 

code for displaying a first lens emphasizing a portion of the first section of the 
first area occurring between a first time (ti ) and a second time 02) and a portion of the second 
section of the first area occurring between t] and t2, the emphasized portion of the first section of 
the first area comprising a second set of one or more video keyframes extracted from the video 
information occurring between t\ and t2, the emphasized portion of the second section of the 
first area comprising text information corresponding to information of the first type occurring 
between t\ and t2, wherein the second set of one or more keyframes is a subset of the first set of 
one or more keyframes and (t s ti < t2 t e ); 

code for displaying the second set of one or more keyframes in a first section of a 
second area of the GUI; and 

code for displaying text information corresponding to the information of the first 
type occurring between t\ and t2 in a second section of the second area of the GUI. 

68. (Original) The computer program product of claim 67 further comprising: 
code for displaying a second lens emphasizing a portion of the first section of the 
second area and a portion of the second section of the second area, the emphasized portion of the 
first section of the second area comprising a third set of one or more video keyframes extracted 
from the video information occurring between a third time (ii,) and a fourth time (14), the 
emphasized portion of the second section of the second area comprising text information 
corresponding to information of the first type occurring between 13 and 14, wherein the third set 
of one or more video keyframes is a subset of the second set of one or more video keyframes and 
(ti t 3 <t 4 t 2 ); 
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code for displaying a keyframe from the third set of one or more keyframes in a 
first section of a third area of the GUI; and 

code for displaying text information corresponding to the information of the first 
type occurring between t3 and 14 in a second section of the third area of the GUI. 

69. (Original) The computer program product of claim 67 further comprising: 
code for displaying a second lens emphasizing a portion of the first section of the 

second area and a portion of the second section of the second area, the emphasized portion of the 
first section of the second area comprising a third set of one or more video keyframes extracted 
from the video information occurring between a third time (t3) and a fourth time (t 4 ), the 
emphasized portion of the second section of the second area comprising text information 
corresponding to information of the first type occurring between t3 and t 4 , wherein the third set 
of one or more video keyframes is a subset of the second set of one or more video keyframes and 
(tl t 3 <t 4 t 2 ); 

code for outputting video information starting from t 3 or from t 4 or from a time 
between t 3 and t 4 in a first section of a third area of the GUI; and 

code for displaying text information corresponding to the information of the first 
type occurring between t 3 and t 4 in a second section of the third area of the GUI. 

70. (Original) The computer program product of claim 67 wherein the 
information of the first type is audio information, and the text information corresponding to the 
information of the first type is obtained from transcribing the audio information. 

71 . (Original) The computer program product of claim 67 wherein the 
information of the first type is closed-caption (CC) text information, and the text information 
corresponding to the information of the first type is extracted from the CC text information. 
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72. (Original) The computer program product of claim 67 wherein the 
multimedia information stored by the multimedia document further comprises slides information, 
the computer program product further comprising: 

code for displaying, in a third section of the first area of the GUI, a first set of one 
or more slides extracted from the slides information occurring between t s and t e , wherein the 
first lens emphasizes a portion of the third section of the first area comprising a second set of one 
or more slides extracted from the slides information occurring between ti and t2, the second set 
of one or more slides is a subset of the first set of one or more slides; and 

code for displaying the second set of one or more slides in a third section of the 
second area of the GUI 

73. (Original) The computer program product of claim 72 further comprising: 
code for displaying a second lens emphasizing a portion of the first section of the 

second area, a portion of the second section of the second area, and a portion of the third section 
of the second area, the emphasized portion of the first section of the second area comprising a 
third set of one or more video keyframes extracted from the video information occurring between 
a third time (13) and a fourth time O4), the emphasized portion of the second section of the 
second area comprising text information corresponding to information of the first type occurring 
between t3 and 14, the emphasized portion of the third section of the second area comprising a 
third set of one or more slides extracted from the slides information occurring between t3 and 14, 
wherein the third set of one or more video keyframes is a subset of the second set of one or more 
video keyframes, the third set of one or more slides is a subset of the second set of one or more 
slides, and (t\ 13 < 14 12); 

code for displaying at least one keyframe from the third set of one or more video 
keyframes in a first section of a third area of the GUI; 

code for displaying text information corresponding to the information of the first 
type occurring between t3 and 14 in a second section of the third area of the GUI; and 
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displaying at least one slide from the third set of one or more slides in a third 
section of the third area of the GUI. 

74. (Original) The computer program product of claim 67 wherein the 
multimedia information stored by the multimedia document further comprises whiteboard 
images information, the computer program product further comprising: 

code for displaying, in a third section of the first area of the GUI, a first set of one 
or more whiteboard images extracted from the whiteboard images information occurring 
between t s and t e , wherein the first lens emphasizes a portion of the third section of the first area 
comprising a second set of one or more whiteboard images extracted from the whiteboard images 
information occurring between t^ and t2, the second set of one or more whiteboard images is a 
subset of the first set of one or more whiteboard images; and 

code for displaying the second set of one or more whiteboard images in a third 
section of the second area of the GUI. 

75. (Original) The computer program product of claim 74 further comprising: 
code for displaying a second lens emphasizing a portion of the first section of the 

second area, a portion of the second section of the second area, and a portion of the third section 
of the second area, the emphasized portion of the first section of the second area comprising a 
third set of one or more video keyframes extracted from the video information occurring between 
a third time (t3) and a fourth time (t4), the emphasized portion of the second section of the 
second area comprising text information corresponding to information of the first type occurring 
between t3 and 14, the emphasized portion of the third section of the second area comprising a 
third set of one or more whiteboard images extracted from the whiteboard images information 
occurring between t3 and 14, wherein the third set of one or more video keyframes is a subset of 
the second set of one or more video keyframes, the third set of one or more whiteboard images is 
a subset of the second set of one or more whiteboard images, and (t^ 13 < 14 12); 
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code for displaying at least one keyframe from the third set of one or more video 
keyframes in a first section of a third area of the GUI; 

code for displaying text information corresponding to the information of the first 
type occurring between t3 and 14 in a second section of the third area of the GUI; and 

code for displaying a whiteboard images from the third set of one or more 
whiteboard images in a third section of the third area of the GUI. 

76. (Original) A system for displaying multimedia information stored in a 
multimedia document, the multimedia information comprising information of a plurality of types 
including information of a first type and information of a second type, the system comprising: 

a display; 

a processor; and 

a memory coupled to the processor, the memory configured to store a plurality of 
code modules for execution by the processor, the plurality of code modules comprising: 

a code module for displaying a graphical user interface (GUI) on the 

display; 

a code module for displaying, in a first area of the GUI, a representation of 
the multimedia information stored by the multimedia document, the displayed representation of 
the multimedia information comprising a representation of information of the first type and a 
representation of information of the second type; 

a code module for displaying a first lens covering a first portion of the first 

area; and 

a code module for displaying, in a second area of the GUI, a 
representation of multimedia information displayed in the first portion of the first area, the 
representation of multimedia information displayed in the second area comprising a portion of 
the representation of information of the first type covered by the first lens and a portion of the 
representation of information of the second type covered by the first lens. 
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77. (Original) The system of claim 76 wherein the code module for 
displaying the representation of the multimedia information stored by the multimedia document 
in the first area of the GUI comprises: 

a code module for displaying a first thumbnail image in the first area of the GUI, 
the first thumbnail image comprising the representation of information of the first type; and 

a code module for displaying a second thumbnail image in the first area of the 
GUI, the second thumbnail image comprising the representation of information of the second 
type. 

78. (Original) The system of claim 76 wherein the code module for 
displaying, in the second area of the GUI, the representation of multimedia information 
displayed in the first portion of the first area comprises: 

a code module for displaying the portion of the representation of information of 
the first type covered by the first lens in a first panel in the second area of the GUI; and 

a code module for displaying the portion of the representation of information of 
the second type covered by the first lens in a second panel in the second area of the GUI. 

79. (Original) The system of claim 76 wherein the code module for 
displaying, in the second area of the GUI, the representation of multimedia information 
displayed in the first portion of the first area comprises: 

a code module for determining a first time and a second time associated with the 

first lens; 

a code module for displaying, in the second area of the GUI, a representation of 
information of the first type occurring between the first time and the second time associated with 
the first lens; and 

a code module for displaying, in the second area of the GUI, a representation of 
information of the second type occurring between the first time and the second time associated 
with the first lens. 
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80. (Original) The system of claim 76 wherein the plurality of code modules 
further comprises: 

a code module for receiving user input moving the first lens to cover a second 
portion of the first area; and 

responsive to the user input, a code module for automatically changing the 
information displayed in the second area of the GUI such that the representation of multimedia 
information displayed in the second area of the GUI corresponds to the representation of 
multimedia information included in the second portion of the first area. 

8 1 . (Original) The system of claim 76 wherein the plurality of code modules 
further comprises: 

a code module for displaying a second lens covering a first portion of the second 

area; and 

a code module for displaying, in a third area of the GUI, a representation of 
multimedia information corresponding to the first portion of the second area, the representation 
of multimedia information displayed in the third area comprising a portion of the representation 
of information of the first type covered by the second lens and a portion of the representation of 
information of the second type covered by the second lens. 

82. (Original) The system of claim 81 wherein the code module for 
displaying, in the third area of the GUI, the representation of multimedia information 
corresponding to the first portion of the second area comprises: 

a code module for determining a first time and a second time associated with the 

second lens; 

a code module for displaying, in the third area of the GUI, a representation of 
information of the first type occurring between the first time and the second time associated with 
the second lens; and 



Page 33 of 53 



Appl.No. 10/081,129 
Amdt. dated June 19, 2007 

Amendment under 37 CFR 1.116 Expedited Procedure 
Examining Group 2179 



PATENT 



a code module for displaying, in the third area of the GUI, a representation of 
information of the second type occurring between the first time and the second time associated 
with the second lens. 

83. (Original) The system of claim 8 1 wherein: 

the code module for displaying the representation of the multimedia information 
stored by the multimedia document in the first area of the GUI comprises: 

a code module for displaying a first thumbnail image in the first area of 
the GUI, the first thumbnail image comprising the representation of information of the first type; 
and 

a code module for displaying a second thumbnail image in the first area of 
the GUI, the second thumbnail image comprising the representation of information of the second 
type; 

the code module for displaying the representation of multimedia information 
displayed in the first portion of the first area in the second area of the GUI comprises: 

a code module for displaying the portion of the representation of 
information of the first type covered by the first lens in a first panel in the second area of the 
GUI; and 

a code module for displaying the portion of the representation of 
information of the second type covered by the first lens in a second panel in the second area of 
the GUI; and 

the code module for displaying the representation of multimedia information 
corresponding to the first portion of the second area in the third area of the GUI comprises: 

a code module for displaying the representation of information of the first 
type corresponding to the first portion of the second area of the GUI in a first sub-area of the 
third area of the GUI; and 

a code module for displaying the representation of information of the 
second type corresponding to the first portion of the second area of the GUI in a second sub-area 
of the third area of the GUI. 
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84. (Original) The system of claim 8 1 wherein the plurality of code modules 
further comprises: 

a code module for receiving a user input moving the second lens to cover a 
second portion of the second area; and 

responsive to the user input, a code module for automatically changing the 
information displayed in the third area of the GUI such that the representation of multimedia 
information displayed in the third area of the GUI corresponds to the representation of the 
multimedia information included in the second portion of the second area. 

85. (Original) The system of claim 8 1 wherein the plurality of code modules 
further comprises: 

a code module for receiving a user input moving the first lens to cover a second 
portion of the first area; and 

responsive to the user input, a code module for automatically: 

changing the information displayed in the second area of the GUI such 
that the representation of multimedia information displayed in the second area of the GUI 
corresponds to the representation of multimedia information included in the second portion of 
the first area; and 

changing the information displayed in the third area of the GUI such that 
the representation of multimedia information displayed in the third area of the GUI corresponds 
to the representation of the multimedia information included in the second portion of the second 
area. 

86. (Original) The system of claim 81 wherein the plurality of code modules 
further comprises: 

a code module for displaying a sub-lens covering a portion of the first area of the 
GUI corresponding to the first portion of the second area of the GUI covered by the second lens. 
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87. (Original) The system of claim 86 wherein the plurality of code modules 
further comprises: 

a code module for receiving a user input moving the second lens to cover a 
second portion of the second area; and 

responsive to the user input, a code module for automatically changing a position 
of the sub-lens to cover a portion of the first area of the GUI corresponding to the second portion 
of the second area. 

88. (Original) The system of claim 76 wherein: 

the information of the first type corresponds to video information; and 
the representation of the information of the first type comprises one or more video 
keyframes extracted from the video information. 

89. (Original) The system of claim 88 wherein: 

the information of the second type corresponds to audio information; and 
the representation of information of the second type comprises text information 
obtained from transcribing the audio information. 

90. (Original) The system of claim 88 wherein: 

the information of the second type corresponds to closed-caption (CC) text 
information; and 

the representation of information of the second type comprises text information 
included in the CC text information. 

9 1 . (Original) The system of claim 76 wherein the plurality of code modules 
further comprises: 

a code module for receiving information indicating a user-specified concept of 

interest; and 
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a code module for analyzing the multimedia information stored in the multimedia 
document to identify one or more locations in the multimedia information that are relevant to the 
user-specified concept of interest; 

wherein the code module for displaying the representation of multimedia 
information in the first area of the GUI comprises a code module for annotating the one or more 
locations in the multimedia information that are relevant to the user-specified concept of interest; 
and 

wherein the code module for displaying, in the second area of the GUI, a 
representation of multimedia information displayed in the first portion of the first area comprises 
a code module for annotating the one or more locations in the multimedia information that are 
relevant to the user-specified concept of interest and that are located in the first portion of the 
first area. 

92. (Original) The system of claim 76 wherein the plurality of code modules 
further comprises: 

a code module for receiving input indicating selection of a portion of the 
multimedia information occurring between a first time and a second time; and 

a code module for performing a first operation on the portion of the multimedia 
information occurring between a first time and a second time. 

93. (Original) A system for displaying multimedia information stored in a 
multimedia document, the multimedia information comprising information of a first type and 
information of a second type, the system comprising: 

a display; 

a processor; and 

a memory coupled to the processor, the memory configured to store a plurality of 
code modules for execution by the processor, the plurality of code modules comprising: 

a code module for displaying a graphical user interface (GUI) on the 

display; 
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a code module for displaying, in a first area of the GUI, a representation of 
the multimedia information stored by the multimedia document occurring between a start time 
(t s ) and an end time (t e ) associated with the multimedia document, the displayed representation 
of the multimedia information comprising a representation of information of the first type 
occurring between t s and t e and a representation of information of the second type occurring 
between t s and t e , where (t e > t s ); 

a code module for displaying a first lens emphasizing a portion of the first 
area of the GUI, the portion of the first area emphasized by the first lens comprising a 
representation of multimedia information occurring between a first time (ti ) and a second time 
O2), where (t s t\< t2 t e ); and 

a code module for displaying, in a second area of the GUI, the 
representation of multimedia information occurring between t\ and t2, the representation of 
multimedia information displayed in the second area comprising a representation of information 
of the first type occurring between t\ and t2 and a representation of information of the second 
type occurring between t\ and t2- 

94. (Original) The system of claim 93 wherein the plurality of code modules 
further comprises: 

a code module for displaying a second lens emphasizing a portion of the second 
area of the GUI, the portion of the second area emphasized by the second lens comprising a 
representation of multimedia information occurring between a third time (13) and a fourth time 
(14), where (t^ 13 < 14 12); and 

a code module for displaying, in a third area of the GUI, the representation of 
multimedia information occurring between 13 and 14, the representation of multimedia 
information displayed in the third area comprising a representation of information of the first 
type occurring between t3 and 14 and a representation of information of the second type 
occurring between t3 and 14. 
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95. (Original) The system of claim 94 wherein the plurality of code modules 
further comprises: 

a code module for changing the position of the first lens in response to user input 
such that the first lens emphasizes a portion of the first area of the GUI comprising a 
representation of multimedia information occurring between a fifth time (t5) and a sixth time 
(t 6 ), where (t s t 5 < t 6 t e ), (t 5 t ,), and (t 6 t 2 ); and 

responsive to the change in the position of the first lens, a code module for 
automatically displaying, in the second area of the GUI, the representation of multimedia 
information occurring between t5 and tg, the representation of multimedia information displayed 
in the second area comprising a representation of information of the first type occurring between 
t5 and tg and a representation of information of the second type occurring between t5 and tg. 

96. (Original) The system of claim 94 wherein the plurality of code modules 
further comprises: 

a code module for changing the position of the second lens in response to user 
input such that the second lens emphasizes a portion of the second area of the GUI comprising a 
representation of multimedia information occurring between a fifth time (t5) and a sixth time 
(t 6 ), where (ti t 5 < t 6 t 2 ), (t 5 t 3 ), and (t 6 t 4 ); and 

responsive to the change in the position of the second lens, a code module for 
automatically displaying, in the third area of the GUI, the representation of multimedia 
information occurring between t5 and tg, the representation of multimedia information displayed 
in the third area comprising a representation of information of the first type occurring between t5 
and tg and a representation of information of the second type occurring between t5 and tg. 

97. (Original) The system of claim 94 wherein the plurality of code modules 
further comprises: 

a code module for displaying a third lens emphasizing a portion of the first area of 
the GUI comprising a representation of multimedia information occurring between t 3 and t 4 
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98. (Original) The system of claim 97 wherein the plurality of code modules 
further comprises: 

a code module for changing the position of the second lens in response to user 
input such that the second lens emphasizes a portion of the second area of the GUI comprising a 
representation of multimedia information occurring between a fifth time (t5) and a sixth time 
(t 6 ), where (t! t 5 <t 6 t 2 ), (t 5 t 3 ),and(t 6 t 4 ); and 

responsive to the change in the position of the second lens, a code module for 
automatically changing the position of the third lens such that the third lens emphasizes a portion 
of the first area of the GUI comprising a representation of multimedia information occurring 
between t5 and tg 

99. (Original) The system of claim 93 wherein: 
the information of the first type is video information; 
the information of the second type is audio information; 

the representation of the information of the first type comprises one or more video 
keyframes extracted from the video information; and 

the representation of information of the second type comprises text information 
obtained from transcribing the audio information. 

100. (Original) The system of claim 93 wherein: 
the information of the first type is video information; 

the information of the second type is closed-caption (CC) text information; 

the representation of the information of the first type comprises one or more video 
keyframes extracted from the video information; and 

the representation of the information of the second type comprises text 
information included in the CC text information. 

101. (Original) The system of claim 93 wherein the plurality of code modules 
further comprises: 
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a code module for receiving information indicating a first topic; and 

a code module for analyzing the multimedia information stored in the multimedia 

document to identify one or more locations in the multimedia information that are relevant to the 

first topic; 

wherein the code module for displaying the representation of the multimedia 
information stored by the multimedia document occurring between t s and t e in the first area of 
the GUI comprises a code module for highlighting the one or more locations in the multimedia 
information displayed in the first area of the GUI; and 

wherein the code module for displaying the representation of multimedia 
information occurring between t \ and t2 in the second area of the GUI comprises a code module 
for highlighting the one or more locations in the multimedia information that occur between 
times ti and t2- 

102. (Original) The system of claim 93 wherein the plurality of code modules 
further comprises: 

a code module for receiving input indicating selection of a portion of the 
multimedia information occurring between a selection start time and a selection end time; and 

a code module for performing a first operation on the portion of the multimedia 
information occurring between the selection start time and the selection end time. 

103. (Original) A system of displaying multimedia information stored in a 
multimedia document on a display, the multimedia information comprising video information 
and information of a first type, the system comprising: 

a display; 

a processor; and 

a memory coupled to the processor, the memory configured to store a computer 

program; 

wherein the processor is operative with the computer program to: 
display a graphical user interface (GUI) on the display; 
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display, in a first section of a first area of the GUI, a first set of one or 
more video keyframes extracted from the video information occurring between a start time (t s ) 
and an end time (t e ) associated with the multimedia document, where (t e > t s ); 

display, in a second section of the first area of the GUI, text information 
corresponding to the information of the first type occurring between t s and t e ; 

display a first lens emphasizing a portion of the first section of the first 
area occurring between a first time (ti ) and a second time (t2) and a portion of the second section 
of the first area occurring between ti and t2, the emphasized portion of the first section of the 
first area comprising a second set of one or more video keyframes extracted from the video 
information occurring between t\ and t2, the emphasized portion of the second section of the 
first area comprising text information corresponding to information of the first type occurring 
between t\ and t2, wherein the second set of one or more keyframes is a subset of the first set of 
one or more keyframes and (t s ti < t2 t e ); 

display the second set of one or more keyframes in a first section of a 
second area of the GUI; and 

display text information corresponding to the information of the first type 
occurring between ti and t2 in a second section of the second area of the GUI. 

104. (Original) The system of claim 103 wherein the processor is operative 
with the computer program to: 

display a second lens emphasizing a portion of the first section of the second area 
and a portion of the second section of the second area, the emphasized portion of the first section 
of the second area comprising a third set of one or more video keyframes extracted from the 
video information occurring between a third time (t3) and a fourth time (14), the emphasized 
portion of the second section of the second area comprising text information corresponding to 
information of the first type occurring between 13 and ty, wherein the third set of one or more 
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video keyframes is a subset of the second set of one or more video keyframes and (t^ 13 < 14 

t2); 

display a keyframe from the third set of one or more keyframes in a first section 
of a third area of the GUI; and 

display text information corresponding to the information of the first type 
occurring between t3 and 14 in a second section of the third area of the GUI. 

105. (Original) The system of claim 103 wherein the processor is operative 
with the computer program to: 

display a second lens emphasizing a portion of the first section of the second area 
and a portion of the second section of the second area, the emphasized portion of the first section 
of the second area comprising a third set of one or more video keyframes extracted from the 
video information occurring between a third time (t3) and a fourth time (t4), the emphasized 
portion of the second section of the second area comprising text information corresponding to 
information of the first type occurring between t3 and t4, wherein the third set of one or more 
video keyframes is a subset of the second set of one or more video keyframes and (t\ < t4 

t 2 ); 

output video information starting from t3 or from t4 or from a time between t3 
and t4 in a first section of a third area of the GUI; and 

display text information corresponding to the information of the first type 
occurring between t3 and t4 in a second section of the third area of the GUI. 

106. (Original) The system of claim 103 wherein the information of the first 
type is audio information, and the text information corresponding to the information of the first 
type is obtained from transcribing the audio information. 
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107. (Original) The system of claim 103 wherein the information of the first 
type is closed-caption (CC) text information, and the text information corresponding to the 
information of the first type is extracted from the CC text information. 

108. (Original) The system of claim 103 wherein the multimedia information 
stored by the multimedia document further comprises slides information, and wherein the 
processor is operative with the computer program to: 

display, in a third section of the first area of the GUI, a first set of one or more 
slides extracted from the slides information occurring between t s and t e , wherein the first lens 
emphasizes a portion of the third section of the first area comprising a second set of one or more 
slides extracted from the slides information occurring between t] and t2, the second set of one or 
more slides is a subset of the first set of one or more slides; and 

display the second set of one or more slides in a third section of the second area of 

the GUI 

109. (Original) The system of claim 108 wherein the processor is operative 
with the computer program to: 

display a second lens emphasizing a portion of the first section of the second area, 
a portion of the second section of the second area, and a portion of the third section of the second 
area, the emphasized portion of the first section of the second area comprising a third set of one 
or more video keyframes extracted from the video information occurring between a third time 
(t3) and a fourth time (14), the emphasized portion of the second section of the second area 
comprising text information corresponding to information of the first type occurring between t3 
and t4, the emphasized portion of the third section of the second area comprising a third set of 
one or more slides extracted from the slides information occurring between t3 and t4, wherein 
the third set of one or more video keyframes is a subset of the second set of one or more video 
keyframes, the third set of one or more slides is a subset of the second set of one or more slides, 
and (4 t 3 <t 4 t 2 ); 
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display at least one keyframe from the third set of one or more video keyframes in 
a first section of a third area of the GUI; 

display text information corresponding to the information of the first type 
occurring between t3 and 14 in a second section of the third area of the GUI; and 

display at least one slide from the third set of one or more slides in a third section 
of the third area of the GUI. 

110. (Original) The system of claim 103 wherein the multimedia information 
stored by the multimedia document further comprises whiteboard images information, and 
wherein the processor is operative with the computer program to: 

display, in a third section of the first area of the GUI, a first set of one or more 
whiteboard images extracted from the whiteboard images information occurring between t s and 
t e , wherein the first lens emphasizes a portion of the third section of the first area comprising a 
second set of one or more whiteboard images extracted from the whiteboard images information 
occurring between t\ and t2, the second set of one or more whiteboard images is a subset of the 
first set of one or more whiteboard images; and 

display the second set of one or more whiteboard images in a third section of the 
second area of the GUI. 

111. (Original) The system of claim 1 1 0 wherein the processor is operative 
with the computer program to: 

display a second lens emphasizing a portion of the first section of the second area, 
a portion of the second section of the second area, and a portion of the third section of the second 
area, the emphasized portion of the first section of the second area comprising a third set of one 
or more video keyframes extracted from the video information occurring between a third time (t3 
) and a fourth time (14), the emphasized portion of the second section of the second area 
comprising text information corresponding to information of the first type occurring between t3 
and 14, the emphasized portion of the third section of the second area comprising a third set of 
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one or more whiteboard images extracted from the whiteboard images information occurring 
between t3 and ty, wherein the third set of one or more video keyframes is a subset of the second 
set of one or more video keyframes, the third set of one or more whiteboard images is a subset of 
the second set of one or more whiteboard images, and (ti t3 < t4 t2); 

display at least one keyframe from the third set of one or more video keyframes in 
a first section of a third area of the GUI; 

display text information corresponding to the information of the first type 
occurring between t3 and in a second section of the third area of the GUI; and 

display a whiteboard images from the third set of one or more whiteboard images 
in a third section of the third area of the GUI. 



Page 46 of 53 



